Skip to main content

Full text of "Raw Thought [pdf]"

See other formats

Rewriting Reddit 

Translations: Srpsko-Hrvatski 

20 12 note: This article was first published in 2005. After it was published, Django launched a RemovingTheMagic project to address some of my criticisms 
(though personally I still find it unusable), inspired FriendFeed's tornado. web and Google's gae.webapp and others (though I still prefer webpy), and this 
article led to a permanent surge in Reddit traffic that still hasn't really stopped growing. 

Over at , we rewrote the site from Lisp to Python in the past week. It was pretty much done in one weekend. (Disclosure: We used my library.) 
The others knew Lisp (they wrote their whole site in it) and they knew Python (they rewrote their whole site in it) and yet they decided liked Python better for this 
project. The Python version had less code that ran faster and was far easier to read and maintain. 

The idea that there is something better than Lisp is apparently inconceivable to some, judging from comments on the reddit blog . The Lispers instead quickly set 
about trying to find the real reason behind the switch. 

One assumed it must have been divine intervention, since "there seems to be no other reason for switching to an inferior language." Another figured something 
else must be going on: "Could this be... a lie? To throw off competition? It's not as though Paul Graham hasn't hinted at this tactic in his essays..." Another 
chimed in: "I decided it was a prank." Another suggested the authors simply wanted more "cut corners, hacks, and faked artisanship." 

These were, of course, extreme cases. Others assumed there must have been outside pressure. "Either libraries or hiring new programmers I guess." Another 
concluded: "some vc suit wants a maintainable-by-joe-programmer product. I hope he pays you millions." 

The Lisp newsgroup, comp . lang .lisp, was upset about the switch that they're currently planning to write a competitors reddit in Lisp , to show how right they 
are or something. 

The more sane argued along the lines of saying Lisp's value lies in being able to create new linguistic constructs and that for something like a simple web app, 
this isn't necessary, since the constructs have been already built. But even this isn't true, was built pretty much from scratch and uses all sorts of "new 
linguistic constructs" and — even better — these constructs have syntax that goes along with them and makes them reasonably readable. Sure, Python isn't Perl 
6, so you can't add arbitrary syntax, but you can often find a clever way to get the job done. 

Python, on the other hand, has problems of its own. The biggest is that it has dozens of web application frameworks, but none of them are any good. Pythonists 
are well aware of the first part but apparently not of the second, since when I tell them that I'm using my own library, the universal response is "I don't think Python 
needs another web application framework". Yes, Python needs fewer web application frameworks. But it also needs one that doesn't suck. 

The framework that seems most promising is Django and indeed we initially attempted to rewrite Reddit in it. As the most experienced Python programmer, I tried 
my best to help the others out. 

Django seemed great from the outside: a nice-looking website, intelligent and talented developers, and a seeming surplus of nice features. The developers and 
community are extremely helpful and responsive to patches and suggestions. And all the right goals are espoused in their philosophy documents and FAQs. 
Unfortunately, however, they seem completely incapable of living up to them. 

While Django claims that it's "loosely coupled", using it pretty much requires fitting your code into Django's worldview. Django insists on executing your code itself, 
either through its command-line utility or a specialized server handler called with the appropriate environment variables and Python path. When you start a 
project, by default Django creates folders nested four levels deep for your code and while you can move around some files, I had trouble figuring out which ones 
and how. 

Django's philosophy says "Explicit is better than implicit", but Django has all sorts of magic. Database models you create in one file magically appear someplace 
else deep inside the Django module with a different name. When your model function is called, new things have been added to its variable-space and old ones 
removed. (I'm told they're currently working on fixing both of these, though.) 

Another Django goal is "less code", at least for you. But Django is simply full of code. Inside the django module are 10 different folders and inside each of those 

are a few more. By the time you actually build a site in the Django tutorial, you've imported django. core, met a, django. models, polls, 

django .conf.urls. defaults . *, django . utils . httpwrappers . HttpResponse, and django . core . extensions . render_to_response. It's not 

clear how anyone is supposed to remember all that, especially since there appear to be no guiding principles for what goes where or how it's named. Three of 
these are inserted automatically by the start scripts, but you still need to memorize such names for every other function you want to use. 

But Django's most important problem is that its developers seem incapable of designing a decent API. They're clearly capable Python programmers — their code 
uses all sorts of bizarre tricks. And they're clearly able to write code that works — they have all sorts of interesting features. But they can't seem to shape this 
code into something that other people can use. 

Their APIs are ugly and regularly missing key features: the database API figures out queries by counting underscores but has no special syntax for JOINs, the 
template system requires four curly braces around every variable and can't do any sort of computation, the form API requires 15 lines to process a form and can't 
automatically generate the template. 

I tried my best to fix things — and the Django community was extremely supportive — but the task simply dwarfed me. I just couldn't do it mentally, let alone with 
the time constraints of having to actually build my own application for my own startup. 

And so, Lisp and Django found wanting, we're left with . I'd like to say that learned from these mistakes and was designed to avoid them, but the 
truth is that was written long before all this and managed to avoid them anyway. 

The way I wrote was simple: I imagined how things should work and then I made that happen. Sometimes making things just work takes a lot of code. 
Sometimes it only takes a little. But either way, that fact is hidden from the user — they just get the ideal API. 

So how should things work? The first principle is that code should be clear and simple. If you want to output some text, you call web. output. If you want to get 
form input, you call web. input. There's nothing particularly hard to remember. 

The second principle is that should fit your code, not the other way around. Every function in is completely independent, you can use whichever 
ones you want. You can put your files wherever you like, and will happily follow along. If you want a piece of code to be run as a web app, you call 
web. run, you don't put your code in the magical place so that can run you. 

The third principle is that should, by default, do the right thing by the Web. This means distinguishing between GET and POST properly. It means simple, 
canonical URLs which synonyms redirect to. It means readable HTML with the proper HTTP headers. 

And that, as far as I'm concerned, are pretty much all the principles you need. They seem pretty simple and obvious to me and I'm even willing to fudge on some 
of them, but no other Python web app framework seems to even come close. (If you know of one, tell me and I'll happily recant. I don't want to be in this 
business.) Until then, it looks like I'm forced to do that horrible thing I'd rather not do: release one more Python web application framework into the world. 

You should follow me on twitter here . 

December 6, 2005